Hallucination: A Mixed-Initiative Approach for Efficient Document Reconstruction

We introduce a mixed-initiative approach for document reconstruction that can significantly reduce the amount of time and effort required to reassemble a document from shredded pieces or an artifact from broken fragments. We focus in particular on the hardest subproblem, which is the problem of identifying a matching neighbor for any given piece. Our approach, called hallucination, combines human and machine intelligence by leveraging people’s ability to draw what a neighboring piece may look like, and then using the drawing as a template based on which the computer computes likely matches. Experiments on a puzzle from the DARPA Shredder Challenge demonstrate that the hallucination approach significantly reduces the search space for identifying a match, outperforming humans and computers working in isolation.